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As students graduate and enter the workforce, they face the job market’s demand for critical thinking 
(CT) skills. The demand is caused by the market’s increasing need for providing professional 
services that require performing complex tasks. In response to this demand, institutions of higher 
education are expected to prepare their graduate through incorporating courses in their curricula that 
promote CT skills. While the definition of CT is contested across various scientific fields, several 
approaches to designing CT-based instruction have been proposed. This paper presents an 
application case of “immersion” and “infusion” approaches, borrowed from Ennis (1989), to a 
graduate course on evaluation of training and examines the results in terms of the critical thinking 
VALUE rubric developed by the American Colleges and Universities (AACU). We contend that 
successful application of these approaches depends heavily on relevant complex scaffolds that 
induce learners’ immersion in CT and allow infusion of instructional features that support their CT 
activities. In our case, we used Systems Thinking to scaffold learners’ immersion and adopted 
Human Performance Technology (HPT) to infuse learning activities aimed at CT. We finally 
examined our procedures and outcomes by using the AACU Value Rubric milestones. 


The emphasis on cultivating critical thinking (CT) 
skills in students across all ages has been growing in the 
past decade. Educational standards for K-12 education 
emphasize improved CT as an outcome (e.g., Common 
Core Standards and 21 st Century Skills), and it is also 
relevant during and after postsecondary education. Hart 
Research Associates (2013) documented that the job 
market expects higher education institutions to place 
more emphasis on training student competencies that 
lead to five key learning outcomes “including: critical 
thinking complex problem-solving, written and oral 
communication, and applied knowledge in real-world 
settings” (p. 1). Reasons for this demand include the 
changing nature of jobs due to advances in technology, 
which require employees capable of thinking critically 
and possessing transferrable skills to be used 
throughout their careers (Sternberg, 2013). To be able 
to respond to this demand, institutions of higher 
education should address these issues in their curricula 
and apply relevant instructional strategies in graduate 
and undergraduate courses to cultivate the required 
skills. The challenge appears to be more significant in 
graduate programs where students are preparing for 
recruitment by professional organizations to perform 
complex cognitive tasks. 

Since the dawn of the last century, starting with 
scholars such as Dewey (1910), learners’ passive 
acceptance of new information has been considered an 
educational problem. Instead, training reflective thinking 
in writing and critical scrutiny of new information have 
since been recommended as a main purpose of education. 
From this perspective, critical thinking occurs when 
learners investigate the issues and look for new evidence 
to support or counter the claim (Dewey, 1910). Scholars’ 
emphasis on learners’ reflective development led to 
discussion of learners’ cognitive processing of new 


information and classification of learners’ cognitive 
activities in Bloom’s taxonomy (Bloom, Engelhart, Furst, 
Hill, & Krathwohl, 1956). In their original approach to the 
learning cognition process, Bloom and colleagues 
classified the objectives of learning into six hierarchical 
categories: knowledge, comprehension, application, 
analysis, evaluation, and synthesis (Bloom et al., 1956). 
Later, Krathwohl (2002), revisited this categorization and 
revised the taxonomy objectives according to the typology 
of required knowledge (factual, conceptual, procedural, 
and metacognitive). Krathwohl relabeled the last objective 
as “creating” and included the fundamental skills for 
higher order thinking in the last three categories of the 
taxonomy: analyze, evaluate, and create (Kennedy, 

Fisher, & Ennis, 1991; Lai, 2011). 

Expanding the discussion of learners’ cognitive 
processes, Elder and Paul (1996a), propose the CT 
stage theory according to which learners start as 
unreflective thinkers whose thinking gets challenged, 
which turns them into beginning thinkers who keep 
practicing thinking and advance their thinking skills 
until they master the thinking process. Learners 
progress through these six stages by using a rigorous 
self-assessment while encountering their own incorrect 
beliefs and develop as a thinker (Elder & Paul, 1996b). 

The underpinnings of these discussions on learners’ 
reflection and thinking processes are the foundational 
elements of CT as identified by other authors (see Lai, 
2011). However, some consider using Bloom’s taxonomy 
as a tool for operationalizing CT attributes due to their 
relationship (Miri, David, & Uri, 2007) even though 
interchangeable use of higher order skills and CT is 
considered an incomplete and simplified approach by other 
scholars (Ennis, 1985; Paul, 1990). 

CT is defined differently within the domains of 
cognitive psychology, philosophy, and education (Lai, 
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2011). The common thread among the three fields is the 
use of higher order thinking skills (Critical Thinking 
Community (CTC), 2015; Ennis, 1995; Willingham, 
2007). Cognitive psychology focuses on recognizing 
the intricacies behind an issue, looking for evidence, 
basing one’s beliefs on facts and evidence, and being 
open to ideas different from one’s own beliefs 
(Willingham, 2007). Those in philosophy use the 
cognitive psychological definition while including 
elements or reflective thinking and reasoning in what 
one does (CTC, 2015; Ennis, 1995). 

In the education domain, the Association of 
American Colleges and Universities (AACU) defined 
CT as “a habit of mind characterized by the 
comprehensive exploration of issues, ideas, artifacts, 
and events before accepting or formulating an opinion 
or conclusion” (AACU, 2010). This definition was then 
operationalized through development of a CT rubric 
included in the VALUE (Valid Assessment of Learning 
in Undergraduate Education) rubrics. Both of these 
products, the CT definition and its rubric, were the 
results of a collaborative effort of a group of faculty 
experts from universities across the United States who, 
at the request of AACU, reviewed existing CT 
materials, consulted other faculty, and examined 
existing assessment rubrics for CT (AACU, 2010). 
AACU developed five phases of learners’ critical 
thinking with corresponding indicators for achieving 
the VALUE “milestones.” Following is a paraphrased 
list of AACU’s five phases of CT and their 
corresponding milestones as expected of learners in 
each phase (AACU, 2010): 

1. Explanation of issues. Learners describe 
comprehensively the issues to be considered 
critically by delivering all relevant information 
necessary for full understanding. 

2. Evidence: Learners systematically analyze 
assumptions and carefully examine the 
relevance of context. 

3. Influence of context and assumptions: 

Through questioning the experts’ viewpoints, 
learners evaluate and use information from a 
variety of sources to conduct an analysis or 
synthesis. 

4. Student's position: Learners consider the 
complexities of issues when describing their 
perspective, acknowledge their own 
limitations, and include others’ perspectives 
into their hypothesis. 

5. Conclusions and related outcomes: Learners 
state the logical conclusions, consequences, 
and implications to reflect their informed use 
of prioritized evidence. 
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Given the complexity of defining and 
operationalizing critical thinking, integrating 
instructional strategies in a course or a curriculum 
aimed at promoting the relevant cognitive processes 
presents a pedagogical challenge. In instructional 
systems, the challenge represents itself as selecting one 
overall approach to designing a course with relevant 
learning content and instructional strategies focused on 
advancing critical thinking. Ennis (1989) offers four 
options as overall approaches to developing a course 
with CT in mind. In what he calls a general approach to 
instruction, Ennis recommends designing a course 
specifically devoted to teaching of and training in CT 
skills. In his immersion and infusion approaches, the CT 
skills are integrated in course content, implicitly or 
explicitly respective to the approaches. Ennis’ mixed 
approach combines the general approach with elements 
of either immersion or infusion. In a meta-analysis of 
application of these approaches conducted by Abrami et 
ah, (2008), they found the mixed approach to be most 
effective in teaching CT skills and the immersion 
approach as the least successful. 

This paper presents an application of Ennis’ 
immersion and infusion approaches combined to a 
graduate course on evaluation of training programs. The 
course is offered every other semester as a required 
course for Master’s level students preparing to work in 
the business and industry as practitioners of 
instructional design, development, and evaluation. 
Doctoral students sign up for the course as an elective 
for research and application experience. Both groups 
may use the course to fulfill the requirement of earning 
a certificate in Human Performance Technology. The 
number of students enrolling varies between 9 to 15, 
depending on the students’ schedules and priorities. The 
course content and instructional strategies offer the 
theoretical foundation of evaluation with great 
emphasis on the practical application of investigation 
methods in an authentic environment. Thus, it provides 
a platform for learners’ transition from learning abstract 
evaluation topics to practical use of evaluation methods. 
Due to these features, we found the course to be a good 
fit for a CT-based design of instruction. 

Using immersion and infusion as means of 
applying CT, the instructor used two complex but 
relevant frameworks—systems thinking and human 
performance technology—to scaffold students’ 
processing of CT skills. We integrated various ideas 
and tools from these perspectives into our design and 
asked students to explicitly demonstrate their use of 
these tools in their assignments and presentations aimed 
at promoting CT. In the following section, we present a 
brief description of the two scaffolding frameworks 
before discussing the details of their application. 
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Macro Design Strategies for Immersion and for 
Scaffolding Learners’ CT 

The conventional way of teaching an introductory 
course such as ours usually focuses on teaching “what” 
an evaluation is and “how” to do it. It leaves out the 
learners’ challenging task of addressing the “why” 
elements of the process that require learners’ causal 
reasoning, exploration, and the search for evidence in 
support of their judgment and decision making: all parts 
of CT. Realizing the challenge of provoking and 
facilitating these cognitive processes, we chose to apply 
two relevant macro design strategies to “enable” 
learners to go beyond the declarative and procedural 
knowledge toward a purposeful creation process. To 
accomplish this, in the first few weeks of the course 
students were introduced to Systems Thinking (Ackoff, 
1999; Ghrajedaghi, 1999) and HPT (Stolovitch & 
Keeps, 1999), both of which provided a complex 
cognitive framework that, when applied, demanded 
learners’ high level thinking. These perspectives were 
fundamentally relevant to the course objectives and 
were aimed at contextualizing other instructional 
strategies we used. Both strategies, because of their 
application complexities and intricacies, demanded 
extensive amount of mental effort required for 
developing critical thinking skills. Specifically, the 
application of systems thinking enabled learners’ 
development of analytical skills in order to understand 
the training program’s systemic properties, to 
comprehensively identify and describe the training 
components, and to analyze the relevant training 
information required for appreciation of the program’s 
functions and features. 

Application of HPT principals, on the other hand, 
complemented the systems thinking scaffold in 
enabling students in their evaluation activities, 
requiring higher level CT skills such as synthesizing 
and evaluating their analytical accomplishments. From 
this perspective, a training program is identified as an 
organization with three major functions: 1) improving 
trainees’ knowledge, 2) improving their job 
performance, and 3) contributing to the sponsoring 
organization’s performance as a whole. Focusing on 
these functions, learners’ analysis included examining 
the program’s components contributing to the success 
of these functions and deciding whether there is a need 
for modification of those components based on the 
resulting evidences. 

Infusion Strategies 

To complement using the immersion scaffolds 
aimed at soliciting CT, we “infused” a series of 
sequenced CT-oriented instructional strategies and 
learners’ activities that gradually and incrementally led 
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to production of the course’s capstone project. The 
class activities and assignments were sequenced so that 
students used the immersion scaffolds to describe the 
training program in systems terms and progress through 
the spectrum of critical thinking by identifying, 
analyzing, applying, and synthesizing information to be 
used in creating a proposal. The authenticated 
instructional activities dealt with a real world training 
program and corresponded to theoretical and practical 
features recommended by the literature. 

For the purpose of this paper, we have structured 
our discussion of the immersion and infusion 
application strategies to highlight the relevant 
instructional activities relevant to the AACU’s listed 
milestones for each phase of CT. 

Contributions of Immersion and Infusion Strategies 

Phase 1. Explanation of issues. The CT milestone 
of this phase expects the learners display their abilities 
in explaining the issues under study. The training 
program that our students selected at the outset of the 
course provided a platform for conducting these 
activities that CTC calls an “...intellectually disciplined 
process of actively and skillfully conceptualizing, 
applying, analyzing, synthesizing, and/or evaluating 
information ...generated by, observation, experience, 
reflection, reasoning, or communication, as a guide to 
belief and action” (CTC, 2015, p. 1). The application 
activities were designed for learners to explore and 
explain the functions and features of a real world 
training programs, such as instructional content, 
training strategies, mode of delivery, and training 
environment. The activities were preparatory to the 
foundation of students’ “purposeful, self-regulatory 
judgment” (Facione, 1990) to be used in their decisions 
in designing the evaluation. 

This milestone was achieved when students 
documented their application of systems thinking 
perspectives and contextualized the training program in 
a HPT context (see Figure 1). The students’ cognitive 
efforts for this application and justification of it were 
demonstrated in an essay presented in class and 
submitted to the instructor. In this essay, they described 
how they compared their assigned training program to a 
performance organization as depicted in Figure 1. In 
this organizational system, they identified trainers as 
the workforce that produces a particular product: in this 
case, more knowledgeable trainees with improved 
skills. Students explained how the production occurs 
within a certain organizational structure following 
particular processes that are specifically designed for 
the training purpose. Students also explained how the 
program management and staff, as parts of the program 
environment, and also the sponsoring organization 
observe the transfer of training to the job environment 
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Figure 1 

Training as a system. A systemic view of a training program, identifying key components. 



and provide the feedback about the product. This essay 
is basically a description of Figure 1 as it is applied to 
the student’s selected program. 

Phase 2. Evidencing one’s point of view. Having 
explained the features of their authentic training case, the 
learners identified and selected information for their 
“evidential, conceptual, methodological ... or contextual 
considerations” (Facione, 1990, p. 3). To accomplish this, 
the students used the observation and documentation tools 
integrated into the course for conceptual “deconstruction” of 
the training system and documentation of the components’ 
liinctions. They received instruction on using an adaptation 
of the “Holistic Process of Inquiry,” a systemic analysis 
technique developed by Gharajedaghi (1999). Labeled as 
the “iterative analysis of training systems,” students used 
this tool to examine the training system as depicted in 
Figure 2. They followed this “iterative analysis” procedure 
to evidence their understanding of the selected training 
programs and map the interaction of its components. 
Moreover, this instructional strategy also documented the 
liinctions of the training according to the HPT principles 
which consider the training’s main functions to be 1) 


improving trainees’ knowledge and skills; 2) improving 
trainees’ job performance; and finally 3) improving 
organizational performance (Pershing, 2006). 

We figured that by the end of this analytical process 
students will have gained a deep understanding of their 
selected programs and its operation. Students’ conducted 
observation, collected information, and raised challenging 
questions in their attempt to refine and integrate their 
findings according to their holistic perspectives. 

The achievement of the Phase 2 milestone was 
demonstrated in a class paper they submitted to the 
instructor. In this paper, students’ evidenced their point 
of view in describing their iterative analysis (see Figure 
2) and documented how they reconstructed the system 
according to their views. They described how they 
examined the information about the training program, 
reflected of the program’s functions and features, and 
rationalized how they agreed or disagreed with the 
existing operation of the system. 

Phase 3. Influence of context and assumptions. 
Using the iterative analysis in the previous phase 
allowed the learners to “decompose” the training 
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Figure 2 

Iterative analysis of training system. A tool which guides the analysis of a training program according to its 
functions and helps identify the connections between certain program components. 



system and identify how the components are affected 
by their environments. They also identified and 
examined the assumptions based on which the training 
components are designed and operate. In this phase the 
learners integrated the results of their analysis as 
observation documentations into what the evaluation 
literature calls a “logic model.” 

Developing a logic model or a “logic map” has 
been recommended for helping evaluators define 
“...measurable objectives, a logic or rationale for 
reaching the program’s goals, and a sequence of 
activities that present the program’s logic or rationale” 
(Russ-Eft & Preskill, 2001, p. 90). Holden and 
Zimmerman (2009) describe it as a detailed description 
of the program activities, inputs, outputs, objectives, 
and resources. In our case, the difference is that the 
students develop this model from a systemic 
perspective, connecting the training inputs to the 
training process and concluding with the outputs and 
outcomes of the program as a result. 

Creating this logic model specific to each program 
achieves the Phase 3 milestones in the sense that it 
requires the students’ identification of the program’s 
operation in the organizational context that influences 
every part of the program. Learners’ creation of an 


evaluation logic model (see Figure 3) not only evidenced 
their reflective thinking on the operation of the system, 
but also demonstrated how they influenced the context of 
the program and questioned the assumptions based on 
which the program is operating. The result was 
integration of their own ideas into the development of the 
model which was reported to their peers in class in form 
of a presentation. The logic model is presented as the 
learners’ evidence that in fact the target training program 
is a system with all the systemic features such as input, 
process, output, and outcome. 

Phase 4. Students taking a position. To 
accomplish this milestone, learners were instructed to 
consider the complexities of the evaluation object and 
the concerns of the stakeholders in developing their 
evaluation questions. The questions reflected the 
learners’ hypotheses on how well the training system 
was functioning by focusing on the system’s inputs 
(e.g., instructional content, trainers’ qualifications, 
trainees’ competencies), its training process and outputs 
(e.g., number of sessions, types of classes, 
implementation of its instructional plans and delivery 
method), and finally the short term and long-term 
outcomes of the program. In their formulation and 
presentation of the evaluation questions, students stated 
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Figure 3 

HPT-based Systemic Logic Model for Evaluation of Training Program. This tool breaks down the training program 
based on its inputs, processes, outputs, and outcomes. Students use this template to develop program specific models. 



their positions and the rationale for asking the questions. 
Students also articulated how they synthesized their 
observations to support their questions. 

Following a class lecture on formative and 
summative evaluation, students contemplated and 
formulated relevant evaluation questions (EQs) using 
the “divergent” and “convergent” approach (Worthen, 
Sanders, & Fitzpatrick, 1997). Corresponding to the 


contribution of the training components, the EQs 
focused on the formative attributes of the program (e.g. 
settings, content, trainers, and resources) or its short¬ 
term and long-term outputs and outcomes. 

To indicate their achievement of the Phase 4 
milestone, students integrated their selected questions 
into an Evaluation Management Plan (EMP; See Figure 
4). In this plan, they included qualitative and 
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Figure 4 

Evaluation Management Plan (EMP). A tool used to guide the development of students ’ evaluation proposal, which 
relates the questions directly to the data methods and sources used. 


Program 
Areas of 
focus 

Relevant Training 
Components 

EQs 

(Issues to be examined and 
addressed) 

Type of 
Information to 
Collect 

Qual./Quant./ 

Mixed 

Information Collection Method 
Source 



1 




Training Setup 

2 









Training Process/ 

1 



Formative 

Procedures 

2 



(Process) 


„ 




1 




Training Content 

2 





_ 




Trainers' 

1 




qualification 

2 










1 




Trainees' Readiness 

2 









Organization 

1 




Support 

2 




Resources/ funding 





Short Term: 

1 



Summative 

Individual Learning 

2 



(Outcome) 

(KSA) 

_ 



Long Term: 

1 




Performance Impact 

2 




(on the job /Organizational 

|_ = _| 




quantitative data collection technique depending on the 
type of EQs and identified the sources of information, 
data collection method, and data analysis plan required 
for addressing those questions. In describing EMP, 
rationalizing raising these questions, and developing the 
methodology for addressing them, students basically 
hypothesize their research. Students justified their 
position on their proposed methodology when they 
presented their research method to the class and 
rationalized why they have taken their specific approach. 
In a critique session, instructor and student peers 
discussed and challenged the students’ position on the 
methodology and provided feedback accordingly. 
Through this exercise, students realized that, due to the 
realities of the target program and their use of mixed 
method research, their research approach and use of 
research methods and tools were unique to their 
particular cases. Thus their results were not generalizable 
to other cases, and certainly they recognized the 
limitations of conducting an authentic investigation. 

Phase 5. Conclusions and related outcomes. The 
aforementioned activities resulted in the creation of the 
components of an evaluation proposal, and these were 
synthesized into the final course product. Through class 
presentations of these components (program 


description, program analysis, and evaluation 
methodology) students received peer feedback, as well 
as learning about peers’ work on the same ideas. These 
experiences, in addition to the instructor’s feedback on 
a more detailed print version of the assignments, 
provided an opportunity for student to reflect on and 
evaluate their work. Following their revisions of the 
products according to the given comments and 
feedback, they individually attended a final review 
session with the instructor to prepare an evaluation 
proposal as their final class project. 

To create this proposal as an indicator of achieving 
Phase 5 milestones, students followed a proposal 
outline to seamlessly integrate their previously 
produced products into one document. They started 
with their description and analysis of their selected 
programs, the only fact-based portions of the proposal. 
They incorporated their proposed program-specific 
logic model, EQs, and EMP followed by proposed 
research methodology for addressing the EQs. Students 
also included a communication plan in which they 
described the type of reports they would use and the 
stakeholders they would target to share the evaluation 
results. Together all of these components formed a 
logical and evidence-based conclusion to student 
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activities and resulted in an evaluation document for 
their specific programs. 

Summary and Discussion 

In support of these pedagogical activities aimed at 
immersing students into a CT or higher order thinking 
processes, we would like to summarize our discussion 
by reiterating how Scriven and Paul (1987) reference 
CT as “...the intellectually disciplined process of 
actively and skillfully conceptualizing, applying, 
analyzing, synthesizing, and/or evaluating information 
gathered from, or generated by, observation, 
experience, reflection, reasoning, or communication, as 
a guide to belief and action’’ (CTC, 2015, p. 1). We 
contend that the course’s instructional content and 
strategies, designed according to the CT skills, promote 
these concepts and prepare graduate students for 
performing a complex task like program evaluation in 
their professional careers. 

The instructional strategies and activities used in 
this course elicited students’ purposeful effort to 
achieve what Dewey (1910) presents as reflective 
thinking, or to investigate the issues and look for new 
evidence to support or counter the claim. The students’ 
accomplishments throughout the course, we believe, 
induced them to go through the cognitive processes for 
knowledge gain, comprehension, application, analysis, 
evaluation, and synthesis as listed in Bloom’s 
taxonomy. The capstone of these achievements was the 
production of an evaluation proposal which in 
particular included the higher level thinking activities. 
We suggest that any duplication of our effort in other 
courses should include consideration of the need for 
strong instructional scaffolds that force students into 
reflective critical thinking while providing an 
incremental sequence of activities and assignments for 
demonstration of those efforts. 

Learners’ achievements, as presented under the 
phases of the AACU (2010) VALUE rubric for critical 
thinking, demonstrate their practical journey through 
the stages that Elder and Paul (1996a) proposed. Even 
though our graduate learners should not be considered 
unreflective or beginners in terms of thinking, as Elder 
and Paul put it, they certainly faced cognitive 
challenges in the application of our strategies and their 
production of class projects. As the sequence of the 
assignments kept building up the cognitive demands, 
students kept practicing thinking and advanced their 
thinking skills until they mastered the process as 
reflected in their accomplishments of the milestones 
and course projects. 

However, we strongly recommend that the 
integration of strategies aimed at promoting CT must 
originate from a more complex scaffold that is relevant 
to the content and conducive to producing the results. 
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In our case the course required an analysis of the 
training program which we chose to contextualize in 
systems thinking, and then we designed few procedures 
accordingly. This eliminated the possibility of learners 
providing a simple description of the program and a 
linear observation of its functions. So did the 
application of HPT principles, which contextualized the 
training program as a performance improvement 
intervention. Both of these strategies and applications 
were completely relevant to the course, and students’ 
prior knowledge gained from foundational courses 
contributed to their understanding and facilitated their 
progress through the cognitive stages. 

Given this discussion, we speculate that replication of 
our efforts in graduate courses may not apply to all graduate 
courses. However, we make these recommendations: 

• Identify a course that demands learners’ 
complex cognitive effort in solving a problem 
and/or producing a tangible product. 

• Make sure that the abstract instructional 
contents have a practical application in the 
field, and find a platform for their application. 

• Identify relevant scaffolds that induce learners 
into thinking critically in producing a complex 
course project. 

• Design incremental class projects that 

collectively lead to the production of the final 
course project, the capstone. 

• Sequence the designed class activities from 
easy to difficult to enable students to acquire 
knowledge for going through the process 

• Integrate the activities described under the 
AACU milestones into the sequence of 
activities while designing the infusion approach. 

• Infuse activities such as class presentations, short 
papers, critiquing sessions, and feedback sessions 
so that learners get challenged by their peers and 
defend their position on the issues they discuss. 

• Follow a theoretical framework such as the ones 
suggested by Bloom’s taxonomy and Elder and 
Paul’s (1996b) CT stage theory to sequence the 
instructional materials and course activities. 
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